Repository navigation
Phase 32 slice A: tests in folders that set their tier, type-checked - #255
Merged
Merged
Conversation
Tests sat loose in the repo root, tiered by a text match that guessed which ones launch Chrome, and were never type-checked. - test/unit, test/browser, test/e2e and test/manual; the runner reads the folders, and runs test/unit with Chrome unlaunchable, naming any unit test that launches it - tsconfig.test.json (strict, noEmit) and `npm run typecheck:tests`, run in CI's fast job; the 54 existing errors fixed by typing the tests - test-phase2/3 split by subject; the two issue-named tests renamed by what they check; evaluate's detailed block moved to test/browser without its skip-on-no-Chrome catch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 32 slice A (
docs/specs/PHASE-32-SPEC.md, FR-A1 to FR-A7). The tests move out of the repo root into folders that set their tier, the runner proves the unit tier never launches Chrome, and the tests are type-checked. No test changes what it asserts, except where noted below.What changed
test/unit(Node only),test/browser(launches Chrome),test/e2e(spawns the MCP server or the CLI) andtest/manual(needs a human or a running viewer).test-env.tsis nowtest/env.ts, andtest-fixtures/is nowtest/fixtures/. Files keep theirtest-names.test:fastrunstest/unit, andtest:fulladdstest/browserandtest/e2e.test/manualnever runs. The text-match tiering (CHROME_MARKERS,CHROME_ANYWAY,INTERACTIVE) is deleted. The runner switches to the repo root itself, so it works from any directory. Filters match a test's path, sounit/selects a folder.--root <dir>points the runner at another folder, which its own tests use.test/unitfile runs withFRAMESMITH_CHROME_PATHandPUPPETEER_EXECUTABLE_PATHset to a path that does not exist. A unit test that launches Chrome therefore fails, and the runner adds a line telling you to move it totest/browser. The newtest/unit/test-runner-guard.tsproves this for a launch throughsrc/screenshot.tsand for a directpuppeteer.launch. Each file was classified by running it under that guard: 88 passed and 40 failed with a launch error.test-evaluate.tsused to catch a failed launch in its detailed-mode block and count it as a pass. That block now lives intest/browser/test-evaluate-detailed.ts, without the catch.tsconfig.test.jsonextends the main config withstrictandnoEmitovertest/**.npm run typecheck:testsruns it, and so does CI's fast job, after the build. The 54 existing errors are fixed by typing the tests, not withanyor@ts-ignore. In practice that meant using the exportedEvaluationIssuetype, adding theprojectIdaCanvasrequires, and writing'end'and'start'where tests had'flex-end'and'flex-start'(both render the same CSS). Feedback and critique fixtures now match their real types, andStressRecordfixtures gained the requiredtype.test-phase2.tsandtest-phase3.tsare split by subject; the table below maps every assertion.test-correctness-fixes.tsis nowtest-contrast-rounding-preset-inherit.ts, andtest-gradient-shadow-move-fixes.tsis nowtest-raw-css-styles-repo-move.ts. Hardcoded/tmpoutputs now useos.tmpdir().test-guidelines.ts, which now derives the repo root from its new depth. Every "Usage" header, every test-to-test comment, the README scripts table, three VISION lines and foursrc/comments name the new paths. Closed specs keep their old paths. The spec's stale-path check now returns nothing.Counts
There were 138 active tests before. There are 143 now: the 136 that moved, plus 5 from splitting the phase files, the detailed-evaluate file, and the new runner-guard test, minus the 2 phase files the splits replace. By folder, that is 90 unit, 43 browser and 10 e2e, with 2 more in manual.
Verification
npx tsc -p tsconfig.test.jsonexits 0.npm run buildis clean.test/(npx tsx ../scripts/run-tests.ts), passes 90 of 90.test/against the installed Chrome, passes 142 of 143. The one failure,unit/test-viewer-watch.ts("global canvas write triggers a reload"), is a timing flake: its file watcher waits a fixed time and missed the event while the full run was under load. It passed in the fast tier and in 5 of 5 runs on its own, and this change only touched its import lines.test/browserfile, the five new ones included, fails under the guard. Everytest/unitfile passes under it.test/unit/test-discoverability.tspasses 160 of 160.npm run typecheck:testson Node 22 and Node 24 (run 44760993199). Commit 0cbc9dd reverts it. That probe commit also picked up an unrelated local.framesmith/workspace.jsonedit, which the revert removes, so the branch's net diff touches no.framesmith/file.test-cliche.ts,test-stress.tsandtest-evaluate.tswere read to confirm they never launch. The first two only importshutdownor a type fromsrc/screenshot.ts. The third launched only in the block that moved out.Assertions from the split files
test-phase2.tstestPresets, 8)test/unit/test-presets-icons.tstest-phase2.tstestIcons, 12)test/unit/test-presets-icons.tstest-phase2.tstestExport, 3)test/browser/test-export.tstest-phase2.tstestComponents, 9)test/browser/test-component-instances.tstest-phase2.tstestComponentBinding, 2)test/browser/test-component-instances.tstest-phase3.tsbackground:property, screenshot captured (testGradients, 8)test/browser/test-gradients-shadows.tstest-phase3.tstestShadowsAndBlur, 7)test/browser/test-gradients-shadows.tstest-phase3.tstestResponsiveBreakpoints, 7)test/browser/test-responsive.ts(breakpointShots)test-phase3.tstestDiffMode, 4)test/browser/test-diff.tstest-evaluate.tstestDetailedMode, 2; the catch that counted a pass without Chrome is removed)test/browser/test-evaluate-detailed.tsWhere every file went
test-animations.tstest/browser/test-animations.tstest-auto-relative.tstest/browser/test-auto-relative.tstest-backdrop-filter.tstest/browser/test-backdrop-filter.tstest-check-drift-url.tstest/browser/test-check-drift-url.tstest-color-system-readback.tstest/browser/test-color-system-readback.tstest-custom-fonts.tstest/browser/test-custom-fonts.tstest-default-font.tstest/browser/test-default-font.tstest-desktop-app-overlay-detailed.tstest/browser/test-desktop-app-overlay-detailed.tstest-desktop-app-rules-detailed.tstest/browser/test-desktop-app-rules-detailed.tstest-desktop-primitives-render.tstest/browser/test-desktop-primitives-render.tstest-element-states-render.tstest/browser/test-element-states-render.tstest-fluid-widths.tstest/browser/test-fluid-widths.tstest-grid.tstest/browser/test-grid.tstest-import-html.tstest/browser/test-import-html.tstest-import-url.tstest/browser/test-import-url.tstest-material-icons.tstest/browser/test-material-icons.tstest-overlays-render.tstest/browser/test-overlays-render.tstest-patterns-desktop-window.tstest/browser/test-patterns-desktop-window.tstest-patterns-desktop.tstest/browser/test-patterns-desktop.tstest-patterns-mobile.tstest/browser/test-patterns-mobile.tstest-pointer-targets.tstest/browser/test-pointer-targets.tstest-positioning.tstest/browser/test-positioning.tstest-primitives.tstest/browser/test-primitives.tstest-render-capture.tstest/browser/test-render-capture.tstest-responsive-reflow.tstest/browser/test-responsive-reflow.tstest-responsive.tstest/browser/test-responsive.tstest-root-centering.tstest/browser/test-root-centering.tstest-scaffold-system.tstest/browser/test-scaffold-system.tstest-state-model-render.tstest/browser/test-state-model-render.tstest-stress-render.tstest/browser/test-stress-render.tstest-style-attr-escape.tstest/browser/test-style-attr-escape.tstest-svg-paths.tstest/browser/test-svg-paths.tstest-sync-drift.tstest/browser/test-sync-drift.tstest-text-props.tstest/browser/test-text-props.tstest-usability.tstest/browser/test-usability.tstest-viewer-compare.tstest/browser/test-viewer-compare.tstest-viewer-surface-browser.tstest/browser/test-viewer-surface-browser.tstest-visual.tstest/browser/test-visual.tstest-cli.tstest/e2e/test-cli.tstest-evaluate-fonts.tstest/e2e/test-evaluate-fonts.tstest-genre-mcp.tstest/e2e/test-genre-mcp.tstest-mobile-rules-detailed.tstest/e2e/test-mobile-rules-detailed.tstest-mobile-rules.tstest/e2e/test-mobile-rules.tstest-ridgeline.tstest/e2e/test-ridgeline.tstest-structures-mcp.tstest/e2e/test-structures-mcp.tstest-surface.tstest/e2e/test-surface.tstest-verify-inherited-tokens.tstest/e2e/test-verify-inherited-tokens.tstest-viewer-rebuild.tstest/e2e/test-viewer-rebuild.tstest-env.tstest/env.tstest-viewer-navbar.tstest/manual/test-viewer-navbar.tstest-viewer.tstest/manual/test-viewer.tstest-advisory-honesty.tstest/unit/test-advisory-honesty.tstest-apply-structure.tstest/unit/test-apply-structure.tstest-archive-view.tstest/unit/test-archive-view.tstest-autofix.tstest/unit/test-autofix.tstest-batch-design-idmap.tstest/unit/test-batch-design-idmap.tstest-borders-dasharray.tstest/unit/test-borders-dasharray.tstest-build-log.tstest/unit/test-build-log.tstest-calibration.tstest/unit/test-calibration.tstest-chart.tstest/unit/test-chart.tstest-charts.tstest/unit/test-charts.tstest-cliche.tstest/unit/test-cliche.tstest-color-system.tstest/unit/test-color-system.tstest-component-structures.tstest/unit/test-component-structures.tstest-components.tstest/unit/test-components.tstest-contrast-alpha.tstest/unit/test-contrast-alpha.tstest-correctness-fixes.tstest/unit/test-contrast-rounding-preset-inherit.tstest-coverage.tstest/unit/test-coverage.tstest-critique.tstest/unit/test-critique.tstest-design-language.tstest/unit/test-design-language.tstest-design-md-tolerant.tstest/unit/test-design-md-tolerant.tstest-design-md.tstest/unit/test-design-md.tstest-design-system-elevation.tstest/unit/test-design-system-elevation.tstest-design-system-inheritance.tstest/unit/test-design-system-inheritance.tstest-desktop-app-overlay.tstest/unit/test-desktop-app-overlay.tstest-desktop-app-rules.tstest/unit/test-desktop-app-rules.tstest-desktop-app.tstest/unit/test-desktop-app.tstest-desktop-primitives.tstest/unit/test-desktop-primitives.tstest-desktop-structures.tstest/unit/test-desktop-structures.tstest-discoverability.tstest/unit/test-discoverability.tstest-diversification.tstest/unit/test-diversification.tstest-drift.tstest/unit/test-drift.tstest-dual-theme.tstest/unit/test-dual-theme.tstest-element-states.tstest/unit/test-element-states.tstest-empty-thumbnail.tstest/unit/test-empty-thumbnail.tstest-eval-cache-key.tstest/unit/test-eval-cache-key.tstest-evaluate.tstest/unit/test-evaluate.tstest-feedback.tstest/unit/test-feedback.tstest-find-nodes.tstest/unit/test-find-nodes.tstest-flow-critique.tstest/unit/test-flow-critique.tstest-fluid-type.tstest/unit/test-fluid-type.tstest-font-aliases.tstest/unit/test-font-aliases.tstest-fonts-by-name.tstest/unit/test-fonts-by-name.tstest-genre-report.tstest/unit/test-genre-report.tstest-guidelines.tstest/unit/test-guidelines.tstest-import-core.tstest/unit/test-import-core.tstest-import-structure.tstest/unit/test-import-structure.tstest-init.tstest/unit/test-init.tstest-llm-judge.tstest/unit/test-llm-judge.tstest-mobile-primitives.tstest/unit/test-mobile-primitives.tstest-motion-tokens.tstest/unit/test-motion-tokens.tstest-ops-parser.tstest/unit/test-ops-parser.tstest-overlays.tstest/unit/test-overlays.tstest-patterns.tstest/unit/test-patterns.tstest-preservation.tstest/unit/test-preservation.tstest-project-evaluate.tstest/unit/test-project-evaluate.tstest-gradient-shadow-move-fixes.tstest/unit/test-raw-css-styles-repo-move.tstest-replace-matching.tstest/unit/test-replace-matching.tstest-repo-assets.tstest/unit/test-repo-assets.tstest-repo-canvases.tstest/unit/test-repo-canvases.tstest-repo-external-change.tstest/unit/test-repo-external-change.tstest-repo-workspace-external-change.tstest/unit/test-repo-workspace-external-change.tstest-runner-timeout.tstest/unit/test-runner-timeout.tstest-scales.tstest/unit/test-scales.tstest-set-genre.tstest/unit/test-set-genre.tstest-shadow-warning.tstest/unit/test-shadow-warning.tstest-silent-drops.tstest/unit/test-silent-drops.tstest-skeleton.tstest/unit/test-skeleton.tstest-state-checks.tstest/unit/test-state-checks.tstest-stress.tstest/unit/test-stress.tstest-structures.tstest/unit/test-structures.tstest-tailwind-map.tstest/unit/test-tailwind-map.tstest-token-lint.tstest/unit/test-token-lint.tstest-typography-depth.tstest/unit/test-typography-depth.tstest-unknown-icons.tstest/unit/test-unknown-icons.tstest-variants.tstest/unit/test-variants.tstest-version-hash.tstest/unit/test-version-hash.tstest-viewer-feedback.tstest/unit/test-viewer-feedback.tstest-viewer-logo-favicon.tstest/unit/test-viewer-logo-favicon.tstest-viewer-panels.tstest/unit/test-viewer-panels.tstest-viewer-perf.tstest/unit/test-viewer-perf.tstest-viewer-repo-aggregation.tstest/unit/test-viewer-repo-aggregation.tstest-viewer-repo-lifecycle.tstest/unit/test-viewer-repo-lifecycle.tstest-viewer-resolves-workspace-tokens.tstest/unit/test-viewer-resolves-workspace-tokens.tstest-viewer-sidebar.tstest/unit/test-viewer-sidebar.tstest-viewer-surface.tstest/unit/test-viewer-surface.tstest-viewer-watch.tstest/unit/test-viewer-watch.tstest-workspace-mcp-tools.tstest/unit/test-workspace-mcp-tools.tstest-workspace-migration.tstest/unit/test-workspace-migration.ts